Generality and transferability. two issues in putting a dialogue evaluation tool into practical use

This paper presents a first set of test results on the generality and transferability of an evaluation tool which can ensure the habitability and usability of spoken dialogues. Building on the assumption that most, if not all, dialogue design errors can be viewed as problems of non-cooperative system behaviour, the tool has two closely related aspects to its use. Firstly, it may be used for the diagnostic evaluation of spoken human-machine dialogue. Secondly, it can be used to guide early dialogue design in order to prevent dialogue design errors from occurring in the implemented system. We describe the development and in-house testing of the tool, and present results of ongoing work on testing its generality and transferability on an external corpus, i.e. an early Wizard of Oz corpus from the development of the Sundial spoken language dialogue system.